MAXIMUM LIKELIHOOD ESTIMATION BY MEANS OP NONLINEAR LEAST SQUARES

Estimation by Least Squares and by Maximum Likelihood

Although statements to the contrary are often made, application of the principle of least squares is not limited to situations in which p is normally distributed. The GaussMarkov theorem is to the effect that, among unbiased estimates which are linear functions of the observations, those yielded by least squares have minimum variance, and the independence of this property from any assumption re...

Least squares and maximum likelihood

Maximum Likelihood Recursive Least Squares Estimation for Multivariable Systems

This paper discusses parameter estimation problems of the multivariable systems described by input–output difference equations. We decompose a multivariable system to several subsystems according to the number of the outputs. Based on the maximum likelihood principle, a maximum likelihood-based recursive least squares algorithm is derived to estimate the parameters of each subsystem. Finally, t...

Nonlinear Least-squares Estimation

The paper uses empirical process techniques to study the asymptotics of the least-squares estimator for the fitting of a nonlinear regression function. By combining and extending ideas of Wu and Van de Geer, it establishes new consistency and central limit theorems that hold under only second moment assumptions on the errors. An application to a delicate example of Wu’s illustrates the use of t...

Least squares methods in maximum likelihood problems

It is well known that the GaussNewton algorithm for solving nonlinear least squares problems is a special case of the scoring algorithm for maximizing log likelihoods. What has received less attention is that the computation of the current correction in the scoring algorithm in both its line search and trust region forms can be cast as a linear least squares problem. This is an important observ...